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Multilevel expander codes 

Alexander Barg* and Gilles Zemor 

Abstract. We define multilevel codes on bipartite graphs which have prop- 
erties analogous to multilevel serial concatenations. A decoding algorithm is 
described that corrects a proportion of errors equal to half the Blokh-Zyablov 
bound on the minimum distance. The error probability of this algorithm has 
exponent similar to that of serially concatenated multilevel codes. 



1. Introduction 

Codes on graphs are presently actively studied both from the standpoint of their 
parameters and the convergence properties of various decoding algorithms. The 
codes on graphs studied in the present work originate in Tanner's paper |14| which 
suggested to index code bits with the edges of the graph and impose local constraints 
on the bits indexed by the edges incident to any given vertex. Following |14| . 
the local constraints are given by a set of parity-check equations of a small error- 
correcting code which may be different for every vertex of the graph. 

In 12 , codes on graphs were shown to correct a proportion of errors growing 
linearly with block length under a simple iterative decoding algorithm whose con- 
vergence region is related to the expansion of the graph. Following 12 , the term 
"expander codes" was adopted for this class of codes. The line of work started 
by |12| was continued in several directions. Studying codes on bipartite graphs, |16| 
introduced a new linear-time iterative decoding algorithm of expander codes which 
provided a better estimate of the fraction of errors correctable by expander codes 
and linked these codes with other concatenated code constructions (this link was 
furthered in [I]). Paper |3J showed that expander codes reach capacity of the binary 
symmetric channel under the iterative decoding algorithm of T!6 . The same paper 
also put forward the idea of using two different codes as local constraints for the two 
parts of the graph, which led to the construction of a family of expander codes that 
match the performance of serially concatenated codes in the sense of Forney "H]. 
Independently, closely related results were obtained in |10| . Code families with im- 
proved parameters and improved estimates of the error probability under iterative 
decoding were also studied in O |2l |5l IllL I13| . 
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The purpose of the present paper is to extend the expander code construction to 
the context of multilevel concatenations in the sense of Blokh and Zyablov [S]. This 
generalization pursues several goals. First, the asymptotic relative distance of the 
expander code family defined below is shown to match the so-called Blokh-Zyablov 
bound 6bz{R) [6], which represents a substantial improvement over the distance- 
rate tradeoff of single- level concatenations I17| I3I. An extension of the decoding 
algorithm of (2j enables one to correct a fraction of errors that is arbitrarily close to 
{1/2)5bz{R)- The same algorithm guarantees a decrease of the error probability of 
decoding which approaches the error exponent of multilevel concatenations of 
The main idea behind the construction of 6 is the use of a tower of nested codes 
in the inner level of concatenation which simultaneously reach the typical perfor- 
mance of random linear codes. Accommodating this idea to the bipartite-graph 
construction and setting up a proper multilevel decoding algorithm are performed 
in a way different from the standard approach. 

The plan of the paper is as follows. Sect.[21is devoted to some basic notation. 
Relevant background from multilevel serial concatenations is reviewed in Sect. |3 
A construction of multilevel parallel concatenations (multilevel expander codes) is 
introduced in Sect. 01 The relative distance of the codes asymptotically approaches 
the Blokh-Zyablov bound. A multistage decoding algorithm of the codes defined 
suggested in Sect. 14.41 corrects a proportion of errors that approaches 5bz{R)/'^ 
in time 0{N), where N is the code length. The error exponent of the algorithm 
approaches the error exponent attainable by the multilevel serial concatenations 
of IS]. 

2. Notation 

We assume transmission over a binary symmetric channel with transition prob- 
ability p, denoted below by BSC (p). Let h{x) = — zloga;— (1 — x) log(l — a;) denote 
the binary entropy function (the base of the logarithms is 2 throughout). Let 
6gv{R) = h~^{l — R) denote the Gilbert- Varshamov (GV) relative distance for the 
rate R. 

Below we will consider probabilities of different events expressed in the form 
exp(— N E) where TV is a large positive number and E a nonnegative function of the 
code parameters. In such situations E will be called the exponent of the probability 
or the error exponent if the event we have in mind is a decoding error. 

Let Eq {R, p) be the "random coding exponent" |2l ■ For i?crit < i? < C we have 
Ea{R,p) = E,p{R,p) D{SGviR)\\p), where 

D{x\\y) x\og{x/y) + (1 - x) log((l - x)/{l - y)), 

^crit = 1 — h{po), Pq — y/p/ (y^ + \/T^^)) is the critical rate and C = 1 — h{p) is 
the channel capacity. For rates < i? < -Rcrit the random coding exponent has the 
following form: 

Eq{R,p) = -SGviR) log2v/p(l-p) iO<R<R^) 

Eo{R,p) = D{po\\p) + Rcrit -R {Rx<R< i?crit), 

where Rx = 1 — /i(2po(l ~ Po))- It is well known that for large code length, typical 
codes from the random code ensemble asymptotically achieve the GV bound and 
have error probability of maximum likelihood decoding behaving as 

exp(-iV(i?o(i?,p)-o(l))). 
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Let q — 2* for some t > 1. Given a binary code A[no, ko — 2*] and a q-aiy code 
B[ni,ki], their concatenation is a hnear code (mapping) defined as follows [1]: 

We write C — ADB to denote the concatenation of codes A and B and note the 
parameters [N = hqUi, K — k^ki] of the code C. 

3. Multilevel serial concatenations 

We begin with recalling the construction and properties of serially concatenated 
multilevel codes. These results are due to Blokh and Zyablov [S]. We review them 
here to emphasize analogies and differences between multilevel serial and parallel 
concatenations. Then we proceed to a description of multilevel concatenations de- 
fined on bipartite graphs, study their parameters and suggest a decoding algorithm. 

3.1. Code construction. A linear mth order concatenated code construction 
involves the following ingredients: a binary [no, ko = uoRq, do] "inner" code A and 
a set of TO "outer" codes Bi, B2, ■ ■ ■ , B^, where the code Bi,i = 1, . . . , to is defined 
over an alphabet of size Qi = 2*». We assume that feg — ^^'^ ^^^^ each of 

the codes B^ has parameters [ni,ki i — niRn, di i]. 

The length of the code C equals N — tiq't-i and its dimension is if = X]i=i ^i.iti- 
The code maps an information sequence u = M2, . . . , uk) to the corresponding 
codeword by first mapping u to a vector in (Fq-^)''^-'^ x • • • x (Fg^)*^! ™ , then encoding 
it with the codes Bi, then mapping the symbols of the obtained codewords back to 
binary digits, and finally encoding them with the code A. The details are as follows: 
let Kq — and for i = 1, . . . , m, let Ki = X]j=i ^ij'^j- Write the data vector u in 
the form u = (ui, . . . ,u™), where u^ = (u^^i, . . . ,Ujj, . . . , Uj,^^ J e (Fa)''!-**' is a 
vector such that 

(see Fig. Next for i ~ l,...,m, map Ui to a /ci.i-vector mi over ¥q. (this 
mapping is an extension of the natural isomorphism) and encode the result with 
the code Bi. We obtain an (mrti)-vector b e F^'^^ x • • • x F^^. For the second 
stage of concatenated encoding, this vector is treated as a vector in where 
F = Fgj X • • • X ¥g^^ , and mapped onto F2 using the isomorphism of additive groups 
F = Fj". Denoting by (•)(, the binary representation of a vector, we can write 
(b)h — (b^, . . . , b"i)fc, where (b')f,, z = 1, . . . , ni is a binary feg-vector. Finally, each 
of these binary vectors is encoded with the code A. The resulting vector c G F2''"^ 
is a codeword of the m-th order concatenated code C. 

The choice of the component codes and decoding of multilevel concatenations 
rely upon a decomposition of the code A into a tower of nested binary codes 

(1) (Fa)"" dA = AidA2D---D Am D Am+i = {0}, 

where Ai is an [^o, X]JLi '^ii '^o.j] binary linear code. Let 

be a generator matrix of A, where Gi,i — 1, . . . ,to is a binary (ti x no)-matrix. 
Then the code Ai,i — 1, ... ,m is generated by the matrix Go,i = (G^ . . . , Gf)'^. 
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Figure 1. Construction of multilevel concatenations: codewords 
of A are read off verticaly, horizontal rectangles of width ti make 
up binary versions of the codes Bi 

Denote by /co.i = X^jli ^^'^ dimension of the code Ai and by Ro^i its rate. 
Note that the rate of the code C equals 

m 

(3) R — ^(i?o,i — Ro,i+l)Rl,i {Ro,m+i ■— 0). 

1=1 

Note also that if all the outer alphabets qt are the same and equal to g = 2*, i.e., 
ti = t for all i, then the code Ai has dimension t{ni ~ i + 1) and the rate of the 
code C can be written as 

(4) 

TO ^ — ' 

i=l 

The distance of the code C is bounded below by d{C) > mini<i<m do,irfi,i- 

A view of multilevel concatenations as direct sums of single-level concatenated 
codes is particularly helpful for a multilevel generalization of expander codes. Let 
us introduce codes A'*) = = 1, . . . ,to, where A^-^^ is generated by G^. 

Then the code A can be represented as a direct sum 

(5) A = A(l)®yl(2)(j...(j^(m)^ 

We have the following theorem (see, e.g., 
Theorem 1. C ^ ^"l^A^'^DBi. 

By definition of the codes A'-'^\ the direct-sum decomposition of the code 
A enables us to write exphcitly the cosets in the quotient spaces Ai/Ai+i,i = 
1, . . . , TO — 1. Namely, let a = (a,„, . . . , ai) be a message vector of the code Ai, 
where a.^ e (F2)** is the corresponding message vector of the code A*^'-' , i = 1, . . . , m. 
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Then the codeword x = aGo,i can be written as 

(6) x= (a„,...,a2)Go,2 + aiGi. 

This representation is useful in decoding of multilevel concatenations. 

Roster of component codes. 

1. The first-level (inner) binary codes 

m 

Ai[no,ko,i = ^tj,do,i], z = f , 2, . . . , m, 

2. The second-level (outer) qi-aiy codes, qt = 2**, 

Bi[ni,kiA,di^i], z=f,2,...,TO, 

3. Binary codes 

A^'^[no,k^'^ =U], z = l,2,...,m. 
These codes are not used directly in the code construction. They are helpful in 
analyzing the decoding. 

We still have to specify which codes are used as components in this construc- 
tion. This choice is related to the complexity restrictions. If the overall objective 
is polynomial construction and decoding complexity, then the codes are chosen as 
follows. We assume that each of the codes Ai,i = 1, . . . , m attains the GV bound, 
i.e., that i?o,i > 1 — ^((Jo.i) — £, and that its error exponent under maximum like- 
lihood decoding approaches the random coding exponent Eo{Ro,i,p)- That such 
a tower of codes exists is established by a standard technique, see [HI Ej- The 
codes Bi, . . . , B,n are taken to be some algebraic q-aiy codes that afford a decoding 
algorithm of complexity polynomial in the length ni, typically Reed-Solomon or 
algebraic geometry codes. 

3.2. Decoding. Decoding proceeds in m stages. Every stage consists of two 
steps, namely, applying maximum likelihood decoding to the code Aj and then 
decoding Bj by an algebraic procedure. 

Let y = yi be the vector received from the channel. We will write yi = 
(yi_i, . . . ,yi,„i), where for every j the vector yij e (F2)"''. In stage one, every 
vector yij,j — l,2,...,7ii is decoded with the code Ai, employing one of the 
possible decoding procedures for this code. Next, for every j, ti information bits 
are extracted from the decoded message sequence (we will assume that the decoding 
algorithm of the codes Ai always outputs a codeword; if not, the procedure is easily 
modified to handle erasures). The ni groups of these message bits, viewed as an ni- 
vector over ^2*1 constitute a "received" word of the code Bi. This word is decoded 
with the code denote the outcome of this decoding by bi = (^1,1, • ■ • i^i.m)- 
Note that if bi ^ hi, the overall decoding procedure has ended in error; this will 
be a part of the error event. 

Stage two (and every subsequence stage) is not much different from stage one; 
however we have now to decode the pair of codes ^2)^2- This requires a transfor- 
mation of the received word yi to a vector y2 = (y2,i, • ■ • ,y2,ni) which relates to 
the code A2 in the same way as yi relates to Ai. We compute 



(7) 



y2j = yij +zij, 
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where zij = {bi,j)bGi, and where {bij)b G (F2)*™ is the binary representation of 
the symbol bij. This transformation is easily understood in the absence of errors: 
in that case this is simply the coset decomposition of the nested codes of (O ■ The 
rest of this decoding stage is the same as in stage one, with {Ai,Bi) replaced with 

(^2,52). 

3.3. Error probability and the choice of parameters. While we have 
chosen the component codes in the construction of C, we still have some freedom 
in choosing the parameters of these codes. This choice is optimized relying on 
performance estimates of decoding. Let us focus on the error probability. Decoding 
the ith level of the code C is not very different from decoding a standard (one-level) 
concatenated code. We assume maximum likelihood decoding of the code Ai and 
Generalized Minimum Distance decoding of the code Bi 8 . The error exponent 
upon decoding of the code Ai can be assumed to approach i?o(i?o,iiP): a-nd the 
overall error exponent obtained upon GMD decoding of the code Bi then equals 
Ei = Eo{Rt)^i,p){l — Ri^i). The overall error exponent for the code C then equals 

(8) E^"'\R,p)^ mm {EoiRQa,p){l-Ria))- 

1 < i < m 

which can be further optimized on the choice of the rates i?o,ij ^i.i given that they 
must satsify relation JSJ. 

Let us assume that all the ti are equal and denote their common value by i, in 
which case i?o,i = t{m — i + l)/no. The following results [Hj are derived under the 
assumption that all the terms under the minimum in (jSJ are the same and equal 
to E^"^^R,p). We then have the following proposition whose proof is included for 
later use. 

Proposition 2. [g] 

Rq — R 



(9) E^'^>iR,p) 



Proof : From ((SJ and the assumptions above, 

E(^-\R,p) 

^^■^^^~~En~5 T (z = l,...,m); 

Eo{Ro^^,p) 

together with Q this gives 



Rm — Rq 



m 

m-i?('")(i?,p) V(£;o(i?o.^,p)) 



1 



Solving for E^"^'^ and using the expression for Rqa, we obtain the claim. I 

For m — I this bound turns into the so-called Forney bound [5] 
(10) E'^'\R,p) = max i?o(i?o,p)(l - R/Ro)- 

For all rates i? G (0, 1), increasing the order of concatenation improves the bound 
on the error exponent, namely 

E'^"'\R,p) < E^"'+^\R,p), m = 1,2,.... 
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Letting to ^ cx) in we obtain the error exponent of concatented codes of 
infinite order: 

(11) E^°-Hr,p)^ max {Ro ^ R) / — . 

R<Ro<C^ 'I Jo Eo{x,p)i 

This is the Blokh-Zyablov exponent |6|. Note an alternative, parametric expression 
for it, obtained upon performing the maximization: 

E^^Hr,p) ^ Eo{a,p) 

r" dx 

R = a~Eo{a,p) — (0 < a < 1 - /i(p)). 

Jo Eo{x,p) 

The convergence of to £;(°°) is uniform for i? e [0, C - e] . In the neighborhood 
of capacity the function under the intergal in has a singularity: a little analysis 
shows that for ijf™) to approximate well E^°°\ the order m has to grow faster than 
1/e. 

3.4. Minimum distance. The minimum (relative) distance of an TO-level 
code for — > cxD approaches the bound [S] 

(12) 5M(i?)= max MRq - R) 

^^^o<iR,jy;i^(Sov{i,Ro)y' 

For m — I this expression turns into the so-called Zyablov bound |17| 

(13) ^z(i?)= max SGviRoKl ~ R/ Ro), 

R<Ro<l 

and for m — > cx) it becomes the Blokh-Zyablov bound, which is easier to write 
expressing the rate i? as a function of the relative distance. We obtain 

dx 

Sgv{x)' 

The Blokh-Zyablov bound can be improved by using long algebraic geometry 
codes as the outer codes Bi,. . . ,B„i. The results have been computed only for 
TO = 1 [T3] . 

3.5. Complexity of multilevel concatenated codes. The decoding com- 
plexity of one-level concatenated codes that meet the Forney bound is O(n^). The 
same complexity estimate is valid for decoding of one-level concatenations that 
correct the proportion of errors that asymptotically approaches Sz- Turning to 
the complexity of m-level concatenations with outer Reed-Solomon codes, we set 
ni — 2^o i"o/'"^ j^Q^ ^ grow as logrio and consider codes of relative distance 5 and 
rate i?, where R and 5 are related by H14(l . Their decoding complexity for correcting 
a 5^BZ proportion of errors, where ^bz is the solution of ((Tl|l with respect to 5, or 
for achieving the error exponent E^°°^ can be bounded above as ni+i°gi°g"/(i~''('5)). 

4. Parallel concatenations 

4.1. Single-level constructions. In its basic version, a bipartite-graph (BG) 
code is defined as follows. Let G{V — VoUVi.E) be a A-regular bipartite graph with 
I Vol = |Vi| = n. Let A[A, i?oA], i3[A, i?iA] be additive binary or g-ary codes. Let us 
fix an ordering of the edges of E and construct a bipartite-graph code C{G\ A, B) 
of length :— \E\ — nA whose coordinates are in a one-to-one correspondence 



(14) Rbz{5) = I - h{6) - 5 
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with the edges m E. For a vector x e C denote by x^, a projection of x on the 
edges incident to a vertex v e V: in other words, if E{v) denotes the set of edges 
incident to v, then x^, = ixj)j^E{v)- 
A vector x is a code vector of C if 

(1) for every w e Vq, the vector e A, 

(2) for every w £Vi, the vector x^ e B. 

The basic iterative procedure used to decode expander codes was introduced 
in |16| and further studied in |2l II 3| . It is described as follows. 

Basic decoding scheme. Given a vector y e {0, 1}^, a left decoding round 
L consists of decoding in parallel with the code A the subvectors for all v E 
Vq. Likewise, a right decoding round R applies decoding with the code B to the 
subvectors for every w € Vi. Decoding of the component codes is assumed 
maximum likelihood. The basic expander decoding scheme consists of performing 
successive decoding steps of the form y.^+i = B.{L{yi)),i = 0,1,.... Decoding 
terminates by either encountering a fixed point or performing O(logn) decoding 
steps. 

In each level of the multilevel construction we will use a modified definition 
of bipartite-graph codes. Introducing this modification enables one to match the 
performance of serial concatenations in the parallel case both for single-level con- 
catenations (Ij and multilevel codes as described below. Let G{V, E) be a bipartite 
graph whose parts are Vq (the left vertices) and Vi U V2 (the right vertices) , where 

= n for « = 0, 1, 2. We will choose both subgraphs Gi = {Vq U Vi,Ei),i = 1, 2 to 
be regular, of degrees Ai and A2 respectively. Thus, the degree of the left vertices 
is A, the degree of the vertices in Vi is Ai, and the degree of vertices in V2 is 
A2 — Ai. Let A be the second largest eigenvalue of the subgraph Gi spanned by 
the vertex sets Vq and Vi. We will assume that A is small compared to Ai, for 
instance, that A — 0(-\/Ai). 

Let us fix an arbitrary ordering of the edges in E. Note that \E\ = nA. We 
are going to construct a linear code C of length N = nAt, where t is an integer 
constant, in such a way that every edge in E corresponds to t coordinates of C. 

For a given vertex v £ Vq we denote by E{v) the set of all edges incident to it 
and by Ei{v) C E{v), i = 1,2 the subset of edges of the form {v, w), where w £ Vi. 
The ordering of the edges on v defines an ordering on Eiiv). 

Let Ahe a. [iA, iJgiA, dg = tAS^] linear binary code of rate Rq = Ai/A. The 
code A can be also viewed as a g-ary additive [A,i?oA] code, (7 = 2*. Let B be 
a q-ary [Ai,i?iAi,di — AiSi] additive code. We will also need an auxiliary g-ary 
code Aaux of length Ai. Every edge of the graph will be associated with t bits of 
the codeword of the code C of length N = ntA. The code C is defined as the set 
of vectors x — {xi, . . . , xa?} such that 

(1) For every vertex v £Vq the subvector {xj)j^E(v) is a (g-ary) codeword of 
A and the set of coordinates Ei{v) is an information set for the code A; 

(2) For every vertex v £ Vi the subvector {xj)j^E{v) is a codeword of B; 

(3) For every vertex v £ Vq the subvector {xj)j^Ei{v) is a codeword of Aaux- 

This code family was introduced in |4j and studied extensively in U |2]. In 
particular, introduced a modified iterative decoding algorithm of the code C 
that uses expansion properties of the graph Gi together with passing reliability 
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information gathered from decoding of the left codes to the right code decoders. 
More precisely, the algorithm is described as follows. 

Modified decoding scheme. Let y e {0, 1}^ be the vector received from 
the channel. In the first step, for every vertex v € Vq, the left decoder computes, 
for every neighboring vertex w € Vi and for every g-ary symbol b, the quantity 
d-{v,w}{b) which is the minimum distance of yy to a codeword of A with symbol b 
in coordinate {v,w}. Then this quantity is passed on through edge {v,w} to the 
right decoder at vertex w. 

In the second step, for every right vertex w, the right decoder finds the codeword 
c = {cj)j^E(w) of B that minimizes '^j^E{w) '^A'^i) (this is an iteration of min-sum 
decoding). The right decoder then writes the symbols of c on its edge set E{w). 

The decoder then reverts to the basic iterative procedure applied to the basic 
bipartite-graph code C((Vo U Vl, Aaux, using the decoding results of the 
second step as its starting values. If this procedure succeeds, then an information 
set of A is recovered at every left vertex, and the whole original codeword x can be 
rederived. 

The complexity of the algorithm is 0{N), similarly to the basic expander de- 
coding scheme. The properties of this algorithm together with the parameters of 
the code family are summarized as follows. 



Theorem 3. |T] The code C has the parameters [N — ntA, RN, D], where 
R> RoRi- Rail- Raux) and 



Let n oo and let R be fixed. For any e > there exists sufficiently large but 
constant values of A and t such that D/N > S{R) — e, where 5{R) is the Zyablov 
bound . The decoding algorithm of [J has error exponent given by (|10|) . The 
algorithm corrects a proportion of errors that approaches (5z(i?)/2. 

Correcting a fraction Sz{R)/2 of errors in linear time was independently ob- 
tained in [lO) . An alternative view of the above code construction was suggested 
in |13| . The approach of |13| also made it possible to use Generalized Minimim 
Distance decoding of [S] in the iterative expander decoding procedure. Thereby 
|13j obtained a different proof of the error correction radius and the error exponent 
for the above code construction. 

4.2. A multilevel construction. Let G be a bipartite graph with the vertex 
set V and the edge set E. The sets V and E are partitioned respectively as 



where Ei is the set of edges between Vq and Vi, i = 1, . . . ,m + 1. The cardinalities 
of the sets Vi, i = 0, . . . ,m + 1, are all taken to be the same, |Vb| = \Vi\ = ... = 
= n, and each edge set Ei, z > 1, is taken to define a regular bipartite 
expander graph d on Vb U of degree A^. Every vertex v £ Vq has therefore 

dcf def 

degree A = Aq = Ai -I- • • • Am -I- Am+i in the resulting graph, and the total number 
of edges equals \E\ — nA. We will assume that each edge in E carries t bits of 




V = U Vi U . . . U F„i U K 

E = _Ei U . . . U Em U Em+l 



111+ 1 
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the codeword of the multilevel BG code C for some positive constant t, so the code 
length is = tnA. 

As above, let us fix an arbitrary order of the edges in E. The edges E{v) 
adjacent to a vertex v G Vq are partitioned into disjoint subsets as follows 

m+l 

E{v) = U {E{v) n E,). 
1=1 

To define the code C, we need several component codes. Let A be a binary linear 
code of length tA and dimension Rot A. Referring to the representation of the graph 
G shown in Figure 1 below, we call this code the "left" code. The code A can be 
also viewed as a g-ary additive code of length A and dimension RqA, where 5 = 2*. 
Suppose again that A affords a nested decomposition 

(15) A = AiDA2D---D A.„, d A„,+i = {0}, 

where this time the codes Ai are viewed as binary or q-ary depending on the context. 
The rate of the code Ai (binary or g-ary) is i?o,i ~ X^Jli ^j/^- '^hc decomposition 
II15|I is chosen to have properties similar to those of in particular, each code Ai is 
assumed to approach the GV bound on the relative distance. The binary version of 
the code Ai is assumed to have error exponent close to the random coding exponent 
Eo{Ro,i,p) under maximum likelihood decoding on the BSC. 
As above, let A be decomposed into a direct sum 

A = ® e . . . e 

where each A^*) is a [tA, tAi] binary linear code. This decomposition is similar 
to the decomposition ©. The codes Ai and A*^*) can be also viewed as g-ary 
additive codes, with obvious adjustments to their parameters. Having the q-aiy 
representation in mind, denote by Gi a generator matrix of the code A^'^ , i — 
1, ... ,771. We will assume that these matrices are chosen in some fixed way. The 
code Ai is generated by Go,i = (Gf , G^^ . . . , G^J^ and therefore can be written 
as a direct sum 

Ai — A*-*-* © ... 

For any c G A^, let c = c^'^ ® • • • ® c*^™) be a direct sum decomposition of c and let 
a-' S ^, j = 7, ... , 777 be the g-ary message vector that corresponds to 0*-^^ i.e., 
the unique vector such that ■ Gj = c^J\ 

We will also need m auxiliary g-ary codes aux of lengths A^ and rates 
-Rauxjij* = 1,..., 777. For each i, the value i?aux,i is assumed to be close to one, 
in particular, i?aux,i = 1 — 0(l/-\/A)- 

Finally, we need m "right" g-ary codes Bi[Ai, Ri^iAi,di,i]. 

Definition 1. An m-level bipartite- graph code C of length N = ntA is a set 
of vectors x G {^2)^ that satisfies the following conditions. 

(1) For every v £ Vb,Xt, e A. 

(2) For every u g Vb, let he the direct-sum decomposition of 
the vector x„. For i — I, . . . ,m,, let the q-ary message vector be defined by the 
equality a{, • Gj = xi''''. Then a*, is a code vector 0/ A,;.aux- 

(3) Let a be the q-ary vector of length uRqA = nJ^^iAi deduced from x by 
writing, for every v £ Vo, the vector aj, on the edge set E{v) D Ei. Then for every 
i = 1, . . . , 771 and every w £ Vi, the vector a^, g Bi. 
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Figure 2. Coding of m- level bipartite-graph codes 

The construction of multilevel bipartite-graph codes is illustrated in Fig. |3 
Roster of component codes. 

1. The "left" binary codes 

ni 

Ai[tA,tko^i = Aj,do,i], I = 1,2, . . . ,TO, 

that form a tower of nested codes l|15|l . These codes can be also viewed as additive 
codes over with parameters [A, ko.i]- 

2. The "right" codes 

Bi[Ai,ki^i = Ri^iAi,di^i = (5i,iAi] over , i = 1, 2, . . . , to, 

3. Binary codes 

AW[iA,iA,], i = l,2,...,m 

such that Ai — ^ A^^^ . The rate of the code A^"^^ equals Rq = Ro,i — Ro.i+i 7 where 

Ro,m+i ■— 0. These codes can be also viewed as additive codes over with the 
parameters [A,Ai]. 

4. 9-ary codes Aj,aux[Ai, i?i,auxAj], ? = 1, 2, . . . , to. 

4.3. Parameters. The rate of the code C is determined in the following 

Proposition 4. For any e > there exists a graph G and a choice of compo- 
nent codes of the code C such that its rate satisfies 

R{C) > ^(-Ro.i ^ Ro,i+i)Ri,i — £, 
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where e ^ 0(l/\/A). 

Proof : From the definition of the code C, 

tnA(l - R{C)) < tnA(l - i?o) + A,(l - i?,,aux) + A,(l - 

i=l i=l 

from which, 

m m 

j=l i=l 

Choosing the values of A^ and the auxiliary codes so that min^ i?i,aux > 1 ^ £/-Roj 
we obtain the claim. I 

Observe that the rate of multilevel parallel concatenations comes close to the 
value of the rate of their serial counterparts (O . 

The distance of the code C is estimated from the distances of expander codes 
supported by the graphs Gi,i — 1, . . . , m. Extending the proof of Theorem|3to the 
m-level case, we obtain 

Theorem 5. For any e > there exist a graph G and a choice of the component 
codes of the multilevel construction such that the distance D of the code C satisfies 

^ > min (5o,i(5i,i - e. 

N l<i<m 

In particular, the results of this section imply that for n — > oo the family of 
multilevel parallel concatenations approaches the Blokh-Zyablov bound dJ. More 
precisely, given an e > and a value of the rate R, it is possible to find large 
(but independent of n) values of A,t,m, a set of codes A,Bi,i = l,...,m, and 
a family of graphs G{„} with n vertices in each component such that as n — * oo, 
the relative distance of the multilevel bipartite graph code will be within e of the 
quantity 5bz(^)- 

4.4. Decoding. Let x be the transmitted codeword and let y = yi be the 
vector received from the channel. For every vertex w S Vb, let — x^ ^ © • • • © x^ 
be the direct sum decomposition of Xt,. The decoding proceeds in m stages, the 
purpose of stage i being the recovery of the vector xi*^ . 

The first stage consists of three steps, similar to the three steps of the modified 
decoding scheme of single-level constructions. 

In the first step, given the vectors yv,v € Vq, the decoder computes for every 
neighboring vertex w G Vi and for every q-ary symbol b, the quantity 
which is the minimum distance of y„ to a codeword 

Cl.tj = c^i}j ® • • • © (^i^J 
of A such that the vector defined by 

a„ • Gi = c(i) 

has symbol b in coordinate {v, w}. Then this quantity is passed on along edge {v, w} 
to the right decoder at vertex w. 

In the second step, for every right vertex w € Vi, the right decoder finds the 
codeword bi — {bij)j^E{w) of the code Bi that minimizes '^j^E{w) ^ji^^j) (^^^^ 
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is an iteration of min-sum decoding) . The right decoder then writes the symbols of 
bi on its edge set E[w). 

The decoder then reverts to the basic iterative decoding scheme apphed to the 
basic bipartite-graph code C((Vb U Vi.Ei); Ai^aux,Bi). If this procedure succeeds, 
then an information vector of A^^^ is recovered at every left vertex, i.e. for every 
w e Vb, the decoder has found such that ■ Gi — x[,"^'. 

Call a^ the vector actually recovered at vertex v at this point of the decoding 
procedure. Next for every € Vb we compute 

(16) y2,v^yv + SilGi. 

The vectors Y2,v form a vector y2 which is submitted to the second stage of the 
decoding procedure. Note that if this first stage of decoding was successful, then 
every vector y2 is such that y2.t, is equal to the channel error vector added to 

x(2) ® •••®x("). 

The second (and every subsequent) stage is now the same as stage one except 
that we use the code A2 to decode y2. Let us elaborate. 

In step one of stage 2 (respectively z), the vectors y2,t, {yi,v) compute, for every 
neighboring vertex w £ V2 {w £ Vi) and for every g-ary symbol b the quantity 
^{■u ii)}(^) ('^{t; tu}(^)) "^l^ich is the minimum distance of y2,^ {yi,v) to a codeword 

C2,D = ® • • • © C2 J (Ci_^ = © • • • ® C-_^^) 

of A2 [Ai) such that the vector sl^ defined as 

a, • G2 = c(2) (a, • G, = c«) 

has symbol h in coordinate {v,w}. This quantity is passed on through edge 
to the right decoder at vertex w. 

In step two of stage 2 (stage i), the right decoder at w G V2 {w ^Vi) writes on 
its edge set E{w) the codeword b2 = {^2,])jeE(w) (bi = {^i,])jf^E(w)) of B2 (B^) 
that minimizes J2jeE(w) ^3(^2^) iJ^jeEiw) djiKj))- 

In step three of stage 2 (stage i), the decoder reverts to the basic decoding 
scheme applied to the basic bipartite-graph code C{{VoLiV2, E2); A2^aux, B2) (resp., 
C((Vb U Vi, Ei)- Ai^aux, Bi)). Call a^ ( a^) the vector recovered at vertex v at the 
end of the basic decoding procedure. For every v € Vq we then compute 

y3,v = y2,v + a^G2 (resp., y^+i.t, = yi^v + a* G^) 

and name ys (yi+i) the vector formed by the y^.y (yi+i,t,), v € Vq. 

The intuition behind the code construction and its decoding is as follows. In the 
zth stage of decoding we would like to perform iterations of expander decoding for 
some expander code relying on the coordinates of the vector output by the {i — l)th 
stage on the coordinates of Ei. This forms a difference with serial concatenations: 
there in the ith decoding stage we did not need to operate separately on the subset of 
coordinates of the code Ai ; for expander codes this is in the core of error correction. 
Note an analogy of ifTCjl with (TJ: every vector yi^^, can be written as yi^v = Ci_„-|-e„ 
where Ci^^, e Ai is the code vector transmitted at the vertex v and e„ is the 
error vector added by the channel. Both procedures and iQ serve the goal of 
revealing a code vector y2,^ = C2,v + Gy, which is in the same relation to the code 
A2 as yi,^ is to Ai. 
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4.5. Performance. The error probability of decoding for multilevel codes on 
bipartite graphs is estimated analogously to the serial case. In particular, the 
probability that the algorithm described in the previous section will result in a 
decoding error in the iih stage, i = 1, . . . , m is estimated in Theorem |31 Choosing 
the component codes as in the one-level parallel construction, we conclude that the 
exponent of this probability approaches HlOfl as n ^ oo. We will choose the code A 
of sufhciently large length A (independent of n) so that every code in the tower of 
codes IjlSf) has error exponent of max-likelihood decoding approaching the random 
coding exponent Eq. 

The complexity of the whole procedure is essentially m times that of the single- 
level case. To estimate the probability of decoding error T), let Vi denote the event 
that decoding is incorrect in the ith stage of the multistage procedure. Then 

= P{Vm{Vrn-lU---UVi)) + P{V^ {Vm-l U • • • U ) 

< P{V„,\ I?„,_iU---Ul?i) + P(I?„_i U • • • U Pi) 

< P{V„,\ I?„_iU---Ul?i) + • • • + P(I?2pi) + P{Vi). 
As remarked above. Theorem |31 implies that for all i = 1, . . . , m, 

p{v,\ x>i_in---nx'i) < 2-"'^*(^o(^o.')(i--Ri.')-=) 

(for i — 1 the condition is empty). Choosing A.; = RoA/m for alH = 1, . . . , m and 
repeating the argument that led to Proposition|21 we obtain the main result of this 
paper. 

Theorem 6. Suppose that the m-level bipartite- graph codes are used on a 
BSC{p). For any rate R < l^h{p) and any e > there exists a family of graphs G of 
sufficiently large degrees A, Ai, . . . , Am and a choice of component codes such that 
the multistage decoding algorithm of Sect. 14.41 /tag the error exponent E^"^')(R,p) — e, 
where E'-"''){R,p) is the error exponent of serially concatenated codes (jHl . For large 
m the error exponent approaches the Blokh-Zyablov bound The proportion of 

errors corrected by the algorithm approaches (1/2)(5bz- 
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